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adult and community learning 
quality support programme 



A 3-year programme to support ACL providers 
to meet quality requirements of inspection and 
funding agencies and improve their provision. It is 
run by the Learning and Skills Development Agency 
(LSDA) in partnership with the National Institute 
of Adult and Continuing Education (NIACE) and is 
funded by the Department for Education and Skills 
(DfES). The programme includes an advice and 
information service, a website, quality improvement 
networks, staff development workshops and consul- 
tancy, development projects and case studies. 
Details of the programme, extra copies of this guide 
and back-up materials are available on the website 
www.qualityACL.org.uk 

Further guides and workshops are planned on: 

■ managing the observation of teaching and learning 

■ equal opportunities 

■ getting learner feedback 

■ involving part-time staff in the quality agenda 

■ fit-for-purpose systems for small providers. 



o 




3 




learning 
and skills 
development 
agency 



observation of teaching and 
learning in adult education 

howto prepare for it, howto 
do it and howto manage it 



David Ewens 



Published by the 

Learning and Skills Development Agency 

www.LSDA.org.uk 

Feedback should be sent to: 

Information Services, 

Learning and Skills Development Agency, 

3 Citadel Place, Tinworth Street, London SE11 5EF. 

Tel 020 7962 1066 Fax 020 7962 1258 
enquiries@LSDA.org.uk 

Registered with the Charity Commissioners 

Editor: Karin Fancett 

Designers: Dave Shaw and Joel Quartey 

Printer: Blackmore Ltd, Shaftesbury, Dorset 

ISBN 1 85338 712 6 

© Learning and Skills Development Agency 2001 

A1165/11/01/3000 

You are welcome to copy this publication 
for internal use within your organisation. 

Otherwise, no part of this publication may 
be reproduced, stored in a retrieval system, 
or transmitted in any form or by any means, 
electronic, electrical, chemical, optical, 
photocopying, recording or otherwise, without 
prior written permission of the copyright owner. 

Note 

The Learning and Skills Development Agency 
was formerly known as FEDA. 

Further information 

For further information on the issues discussed 
in this publication please contact: 

David Ewens, Development Adviser, 

Adult and Community Learning Quality Support Programme, 
Learning and Skills Development Agency, 

3 Citadel Place, Tinworth Street, London SE11 5EF. 

Tel 020 7840 5348 Fax 020 7840 5401 
dewens@LSDA.org.uk 




5 



Contents 



Appendix 1 
Appendix 2 
Appendix 3 



Terms and abbreviations used 


Acknowledgements 


Preface 


Introduction 


1 


Why observe teaching and learning? 


1 


Observing teaching and learning 


3 


Establishing the nature of ‘good practice’ in teaching and 
learning in ACL and developing observation and other forms 


3 


Who observes? 


5 


The process of OTL 


8 


The importance of a protocol in an OTL system 


17 


How a system of observation of teaching and learning 
relates to the SAR 


18 


Implementing a system of teaching and learning observation 


19 


Introducing or adapting an OTL system in the ACL context 


19 


Overview of change management 


20 


Planning change 


20 


A stepped approach to change 


22 


Management styles, strategies and tactics 
in implementing change 


23 


Resistance to change and howto overcome it 


25 


Applying theory to practice 


29 


A case study - Narnshire LEA 


29 


The case study and management of change theory 


38 


Conclusion 


45 


References and bibliography 


47 


Buckinghamshire County Council OTL forms and guidance 


49 


Session plans - original and revised to address CIF questions 


61 


Pre-observation meeting form 


64 




6 



Terms and abbreviations used 

ACL adult and community learning 

ALI Adult Learning Inspectorate 

CIF Common Inspection Framework 

FEFC Further Education Funding Council 

FENTO Further Education National Training Organisation 

LEA local education authority 
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Preface 



This booklet is the second in a series of publications written for the 
adult and community learning (ACL) sector to assist it in improving 
quality. In the introduction and three subsequent chapters it covers 
the following: 

■ the rationale for observing teaching and learning in 
a self-assessment context 

■ observing teaching and learning (the process and related issues) 

a managing change: suggestions for implementing or adapting 
a system of observing teaching and learning 

a applying theory to practice through a case study. 

The first chapter concentrates on the practicalities and processes of 
observing teaching and learning (OTL), including how OTL information 
might be fed into a self-assessment report (SAR). It also suggests 
activities for ‘trainee observers’, and perhaps tutors newly subject 
to observation, that you can follow or adapt to help you carry out 
or revise processes for OTL. The second chapter examines how 
the introduction of a system of OTL might be managed and the 
third chapter applies theory to practice through a case study. 

The third chapter also includes a checklist for change and a way 
of starting calculations to cost a new system and set targets. 

The guide cannot be prescriptive because of the diversity of 
the sector and the many different patterns of delivering adult 
and community learning - either directly through LEAs, through 
partnerships where provision is secured by contract, or by 
combinations of these models. It tries to address the needs 
of all ACL providers more by stimulating consideration of the 
relevant issues than by seeking to be all things to all people. 





Why observe teaching and learning? 

Adult education has always been committed to learners, and the 
relationship between tutors and learners has been central to the 
ethos of adult learning. But how do we know whether learners are 
getting the best possible experiences? There has been a history 
of observing classes, workshops and other teaching and learning 
sessions as part of stage 1 and stage 2 tutor training. Some local 
authorities and large free-standing adult education providers have 
introduced periodic observations as a means of improving the 
quality of provision they fund. But until now there has been little 
external interest in monitoring adult learners’ experiences. Only a 
quarter of LEAs have been inspected by Ofsted since 1993 and only 
those classes funded by the FEFC or under contract to the Training 
and Enterprise Council were covered by FEFC or Training Standards 
Council inspection. 

The situation has now changed. The Learning and Skills Act 2000, 
as well as ‘unifying’ the various strands of the post-16 sector under 
the national Learning and Skills Council (LSC), has put an increased 
emphasis on the importance of learners and their experiences. 

This is highlighted in a number of documents on quality coming from 
the LSC but specifically in Raising standards in post-16 learning: 
self-assessment and development plans ( ALI/ES/LSC/Ofsted 2001, 
page 10, paragraph 33): 

The new arrangements have been designed to ensure 
that the interests of the learner come first and are of 
paramount importance 



i 




9 



with various conditions laid down 

to achieve the objective of ‘placing the learner at the heart 
of the system 

To achieve this objective of putting learners first, the LSC requires 
providers to make improvements in quality by producing annual SARs 
and development plans. This process will be supported by an 
inspection regime underpinned by the CIF agreed by Ofsted and 
the Adult Learning Inspectorate (ALI). The overall aim is not simply to 
prove a certain standard of provision but to demonstrate continuous 
improvement through reflection and action. 

Since the renewed emphasis is on learners and the quality of their 
learning experiences, it is logical that there should be a focus on 
examining and judging quality of learning wherever it takes place. 
Therefore a robust system for observing it is required. 

Two final points conclude this chapter. First, quality is inextricably 
linked with funding. Consistently good quality teaching and learning 
will be rewarded by local LSCs through the ‘provider performance 
review’ process. Satisfactory provision will trigger support in 
devising improvement strategies. Unsatisfactory provision may 
ultimately mean sanctions. OTL is therefore not just about proof 
of quality but also about delivering continuous improvement. 

Second, it is providers who are responsible for the quality of 
their direct provision and that secured by contract to their partners. 

It therefore follows that there should be a uniform OTL system 
across a whole ACL service. 
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Establishing the nature of ‘good practice’ 
in teaching and learning in ACL and 
developing observation and other forms 

When you are setting up a system of OTL you will need to address 
all areas of the teaching and learning cycle - analysing learner need, 
preparing for teaching and learning ( including planning and resource 
development), delivery (teaching, mentoring, coaching, training, 
tutoring, facilitating), assessment and evaluation. To do this, you might 
need to go back to first principles by considering with your team of 
observers what good practice is in ACL and then develop a form for OTL 
on the basis of the good practice you have established. If you already 
have a quality system and an accompanying OTL system, now is a 
good time to adapt it so that it is in line with current requirements. 

Another more ‘root and branch’ approach might be to take the oppor- 
tunity of looking at national standards relating to teaching and learning 
( eg the FENTO standards ) and pick out the ones relevant to classroom 
observations. Some FE colleges have already done this and have based 
observation forms on these standards, but for ACL FENTO standards 
might not be a very good ‘fit’. PAULO standards for community work 
may ultimately be more relevant but are not yet fully developed. 

A useful alternative might be to consider the ‘key questions’ and 
related evaluation and judgement criteria of the CIF that relate 
to teaching and learning. This is the approach of Buckinghamshire 
County Council and has resulted in the ‘Teaching observation report’ 
form in Appendix 1. This also takes account of the ALI's draft 
guidelines on the CIF to ACL providers. One great advantage of 
this approach is that it is relatively straightforward in that there is 
less bulky detail in the CIF than in some of the national standards. 
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Another advantage is that the information that emerges from the 
observations that go towards SARs will closely match what the ALI 
will be looking for when one of its teams comes to inspect. In effect, 
everyone will be talking the same language - inspectors, managers 
and, if the OTL system has been skilfully implemented, tutors. 
‘Ownership’ of the system is very important to its success, 
a theme that will be considered in later chapters. 

Activity 1 on observation forms 
Note 

These exercises are suggestions and can be selected 
and/or adapted according to organisational need. 

1 Split your group of trainee observers into small groups to 
explore what makes up good practice in teaching and learning. 

Use the consensus in a subsequent plenary discussion to develop 
a checklist. 

2 Explore with the group relevant national standards and their use 
in developing observation forms. 

3 Investigate observation forms in the post-16 sector 

( including those used in FE colleges) to inform the development 
or the adoption of a form for your organisation and your partners. 

4 Map the findings of 1, 2 and 3 to the requirements of the CIF. 

5 Select the parts of the key questions of the CIF relevant 
to teaching and learning. 

6 Develop a form for your ACL setting. 



Just as you and your observation team may need to adapt or 
develop observation to take into account new self-assessment and 
inspection requirements, so you may also wish to look at related forms 
and documents, particularly session plans, which should also take 
into account relevant parts of the CIF key questions. Inserting spaces 
for responses to questions such as ‘How will you provide equal oppor- 
tunities for your learners in this session?’, ‘How are you addressing the 
learning styles of each of your learners?’ and ‘How can you arrange 
the classroom (or workshop) environment to maximise learning?’ 
are potentially powerful ‘levers of change’. They will make tutors 
think about these issues before, during and after observations - 
as well as raising staff development issues. In other words, 
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this process may lead you to think about how to implement a training 
programme for observers and tutors so that they can recognise 
the different learning styles among learners - and then address 
their particular learning needs. Appendix 2 shows a basic session 
plan that is then adapted to meet CIF requirements. 

Activity 2 on session plans 

With your ‘trainee observers’ revise your organisation’s standard 
session plan form on the basis of best practice in the post-16 
sector and mapped against the relevant CIF ‘key questions’. 

You will need to address classroom layout, equal opportunities 
and learning styles. 



Who observes? 

We have already referred to your ‘observation team’, but who should 
be in it? Figure 1 identifies who the members of the team might be. 
Various models and approaches have emerged and evolved for 
identifying observers, and different organisations have different 
practices. The issue of grading is dealt with in more detail later 
in this chapter. As far as qualifications are concerned, observers 
should usually have at least a Certificate in Education or equivalent 
and would perhaps gain more credibility from a degree in education. 
Assessor and verifier qualifications are useful because they indicate 
experience in making objective and validating judgements. There are 
many purposes for OTL as well as its part in preparing an SAR on the 
quality of teaching and learning. For example, peer observations, 
if done properly, help both parties and will rarely lead to anything 
but improved quality of teaching and learning. Consider Figure 1 
and, if you have an observation system currently working, decide 
how it might be adapted, improved or expanded. If you are starting 
from scratch, decide what sort of approach would be appropriate 
for your organisation. 
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Figure 1 People who might undertake OTL for self-assessment purposes 



Observers 


Other purposes 


Line managers 


To fit in with appraisal process; 
to satisfy external contractual 
arrangements (eg with an LEA); 
to judge competence issues (but not 
within self-assessment framework) 


Curriculum coordinators; 
tutor organisers 


To provide support within 
a curriculum area (most purposes 
are linked to self-assessment) 


Fellow tutors 


For personal and professional 
development, usually outside 
a curriculum area 



Commissioned consultants 
(including part-time registered 
inspectors) from outside organisations 


To provide external validation of 
an observation system’s results. 
For proof as much as for 
continuous improvement 


Colleagues from partner 
organisations within the LEA 


To provide external validation and 
develop staff from other organisations, 
and to share good practice 


Teacher education staff 


To generate evidence for unqualified 
or partially qualified tutors to gain 
teaching qualifications 
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Issues of status, 
qualifications and credibility etc 


Do they grade ? 


Observer should be sufficiently qualified and 
experienced. Observation by line manager 
may be more stressful than observation 
by others 


Yes, but not necessarily. May use local methods 
linked to performance management review or 
conditions laid down by funder 


Observer should be sufficiently qualified 
and experienced. Observation may be 
less stressful if by than line/centre manager 


Yes, if for self-assessment, but there may be 
other locally agreed criteria. Not necessarily 
compulsory for self-assessment 


Status not so much an issue as mutual respect. 
Credibility will come from good practice and 
the experience and reputation of the tutor. 
Subjectivity might be higher. 

Observation should be less stressful 


Usually not, but possible if accepted as part of 
a self-assessment team or if agreed with tutor. 
The emphasis is on detailed 
qualitative feedback 


Status, qualifications and credibility are 
important issues for an activity to do with 
judgement as much as development. 
Objectivity likely to be high. 

Observation could be stressful 


Yes, but not necessarily - depends on 
the requirements of the organisation 


More important than status is the 
self-development of participants. 
Observation should be less stressful 


Not so likely - but could still provide 
useful evidence for self-assessment reports 


Observation should not be too stressful 
and should be supportive and developmental 


Depends on structure of qualification - 
but still useful for self-assessment 
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The process of OTL 

The actual process of OTL is quite straightforward and 
can normally be split into four distinct parts: 

n preparing for observation ( by both observer and tutor 
being observed) 

■ conducting the observation 

■ discussing the performance, giving feedback and awarding 
the grade ( if grading is going to be a part of your OTL system ) 

■ agreeing and implementing an action plan for organisational 
self-assessment and individual continuing professional 
development purposes. 

Preparing for observation 

Some organisations place more emphasis on this stage than others, 
presumably on the grounds of the four Ps - Poor Planning leads 
to Poor Performance. Pre-observation planning can often involve a 
short meeting between observer and tutor being observed to discuss 
the nature of the course being taught, to examine the session plan, 
scheme of work and assessment schedule, and to look at resources. 
It is also an opportunity to remind both parties of the criteria 
by which the performance is going to be judged. 

Observations themselves are more likely to be successful if both 
parties contribute to making the experience constructive, and 
a pre-meeting can be ideal for this. The tutor can put the lesson or 
learning activity into context. The observer can ask questions and 
get additional information, clarify observation protocols and criteria, 
and sort out feedback times. In the ACL context such a meeting can 
be even more important because there can be less obvious formality 
than in other settings and because there may be an emphasis on 
negotiating learning outcomes with learners. A pre-meeting might 
establish that originally unintended learning outcomes have emerged 
and that aspects of an original course outline or scheme may have 
become redundant or less relevant. Developing a short form for 
use in a pre-observation meeting would be quite straightforward 
( see Appendix 3 ) but might add an unnecessary bureaucratic layer to 
the process. However, it is certainly worthwhile providing summary 
advice (‘Before the visit you should’ ) for observers and tutors, 
as demonstrated in the guidance notes in Appendix 1. 
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Conducting the observation 

Actually undertaking an observation could be considered a beguilingly 
straightforward process. Your observers will be working on your 
previously developed or adapted form, which will have prompts 
for their notes, and the notes will form the basis for feedback. 
Observers will have established in the pre-observation meeting 
what they need to see, and session plan forms will have been 
adapted to meetCIF standards. However, observation and 
report writing do require subtle skills. 

Observers should be trained to stand or sit somewhere that 
provides a good view of tutor and learner activities. This position 
may change as a class continues but observers should try to be 
unobtrusive at all times. They should be able to look at learners’ 
work and speak to them (the latter more normally happening 
outside a session and requiring particular subtlety and sensitivity). 
They should look at registers for attendance and retention patterns. 
It is useful to see a whole tutoring session so that its coherence 
is apparent, including how the beginning relates to the end 
and whether there is a balance of activities. Short of this, the 
pre-observation meeting or agreement should establish what it is 
most fruitful to observe. Naturally, observers should not participate 
or interrupt a tutoring session unless a safety issue arises. 

Observers develop different styles of recording observations. 

Forms with lists of prompts are often very useful for achieving focus. 
However, they can also interrupt the ‘flow’ of thoughts so that vital 
points are missed as the observers make ‘trawls’ of the headings. 

It is often more helpful to have a space on a form to record an 
observation in narrative form. Experienced consultants and 
observers frequently itemise reports into numbered narratives, 
recording interesting or important incidents and interpreting and 
analysing them against the given prompts or framework afterwards. 
In this way, tutors’ distinctive styles and approaches to their work 
are addressed more sympathetically and the reports themselves 
are fresher and therefore more meaningful and useful. 

Quotes, questions and comments recorded roughly word for 
word can help with feedback and lead to a deeper analysis. 

In the end, different observers develop different approaches, with 
increasing experience leading to less reliance on prompts. This is 
surely acceptable so long as there is consistency across the observation 
team in the award of grades. There may be a case for differentiating 
between basic and advanced levels of observer training. 
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Grading observations 

Whether or not to use a grading system is a knotty problem for an 
organisation's senior managers. In Appendix 1, Buckinghamshire 
County Council has opted to grade. The CIF 1-7 grading system 
agreed by Ofsted and ALI (Appendix 1 and ALI/Ofsted 2001b ) might or 
might not become an integral part of the observation process in your 
organisation and its partners. When an ALI inspection team comes to 
inspect ACL provision in the 4 year cycle it will look at your SARs with 
the grades (from 1 to 5) you have awarded for every ‘area of learning’ 
your provision covers and will be interested in how you arrived at them. 
However, this does not automatically mean that your internal 
self-assessment observations have to be graded according to 
the 1 to 7 system. Grading of observations might possibly make it 
easier to establish an area of learning grade by aggregating grades 
and other data into one final grade. However, the qualitative data 
from actual observation reports could easily by themselves contribute 
to the final grade. You might ask yourself, will grading enhance 
the aim of self-assessment and continuous improvement or does 
it detract from the qualitative message of the observation report? 
Excellent grades or poor ones could equally cause tutors to 
gloss over feedback and not hear the rest of the message. 

Whether or not you and your organisation are going to grade, 
your observers need to look at issues such as: 

bj learner need and ability in relation to tutoring and learning activities 

n the appropriateness of session content and approach 

a how challenging the work is 

a the progress that learners make 

( ie the ‘attainment’ referred to in the CIF). 

The last point can be problematic in the context of ACL, particularly 
on non-accredited programmes, and as well as having funding implica- 
tions, it may in itself raise the need for extra observer training. If you 
and your team are going to grade, the highest grade for an observed 
session need not mean that there were no weaknesses - just minor 
ones and substantial strengths. Similarly, the lowest grade signals 
that little learning has taken place. Always get your observers 
to remember that learning is more important than teaching in 
awarding grades ( the former can occur effectively without the latter ! ).. 
In addition, student activity does not necessarily equate with 
learning taking place. Of course, this remains true even without 
a grading system. 
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If you grade, you need to develop thorough criteria for giving 
grades on the CIF scale and they should be explicit for both tutors 
and observers. Identifying strengths and weaknesses deriving 
from assessment should lead to the award of an obviously 
appropriate grade. This is very important indeed so that the system 
is fair and transparent. As things stand, you need to develop your 
own criteria for each grade because at the moment no others exist. 
Activity 3 gives you an opportunity to do this. 

Activity 3 on developing grade criteria 

Using the ALI/Ofsted grade descriptions, work with 
‘trainee observers’ and tutors to develop criteria for each grade 
using the form below and the observation form in Appendix 1. 



Grade 


Criteria for awarding grade 


1 Excellent 




2 Very good 




3 Good 




4 Satisfactory 




5 Unsatisfactory 




6 Poor 




7 Very poor 





Discussing the performance and giving feedback 

Feedback should generally be an affirming and encouraging process. 
Even if an observed session has been very poor, feedback should be 
constructive and used as a learning experience. It should happen soon 
after the observation, when the teaching and learning are fresh in the 
minds of both the tutor and observer, but a little time should be allowed 
for both to reflect. As observers gain experience, there is less need 
for a gap between observation and feedback. If the process cannot 
take place relatively soon after the observation ( ie on the same day), 
there is a very real danger of ‘drift’ leaving feedback not given, a report 
unfinished and the self-assessment process in general undermined. 
It is worth emphasising again to your observers that the key focus is not 
so much on the teaching, tutoring, coaching or facilitation, but on the 
quality of the learning that has taken place. In a grading system the feed- 
back should be firmly and observably related to the grade being given. 
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Active listening skills are vital for observers in the feedback situation. 
Such skills will encourage dialogue and may also clarify episodes 
and interactions in the observation situation. It is often irritating 
and inhibiting to sense that someone is only half attending to what 
you are saying while formulating her or his next sentence or comment. 
In the early stages of self-assessment, recently appointed tutors 
may be accepting of the new requirements. However, there may be 
anxiety and resistance among established tutors, so it is important 
to get all parts of the process right. The feedback stage is perhaps 
the most delicate and important. You should aim for self-assessment, 
and observation and grading of teaching and learning, to be an ‘owned’, 
transparent, supportive and accepted part of the landscape. 

If tutors do not exactly look forward to it with anticipation, 
then at least they will accept its usefulness and validity. 

Observers should not flinch from difficult messages or be apologetic 
about conveying them. However, the notion of a ‘criticism sandwich’ 
is a useful one here. An observer can find something to praise, 
state a criticism unequivocally but then proceed with another item 
of praise. This could be construed as ‘sweetening the pill’, 
but skilfully done it will gain acceptance through reflection. 

There will be times when tutors are not receptive to feedback, but if 
you have managed to introduce the self-assessment procedure and 
accompanying training properly and thoroughly then this problem 
should arise rarely. It is always important, as with difficult learners, 
to concentrate on behaviour rather than the people manifesting 
the behaviour, to be quietly but persistently assertive (and never 
aggressive), to listen actively and to focus on resolution and 
solution rather than problems. A task focus should not preclude 
support for a person. 

There are three possible ways of giving feedback, depending on 
circumstances and the purpose of the feedback. The first approach 
begins with the tutor - asking that person how well a session went. 
This can backfire if the tutor replies ‘really well’ when your judgement 
was distinctly less optimistic! With this approach, it is useful to 
ask open questions that begin a dialogue: ‘How do you think things 
went overall ?’ is probably a better starting point than ‘Do you think 
the learners achieved all the objectives that you established 
between you at the beginning of the session?’ After the dialogue, 
with more focused questions- ‘What about the learning outcomes?’, 
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‘How do you think the resources worked ?’, ‘How could you have 
handled that situation differently?’, ‘What about the questions 
you were asking?’ - observers can then give an analysis of 
strengths and weaknesses, and a grade. 

Either before or after this, observers can cover what worked well 
and what would need further development. An example of 
a comment about what worked well might be: 

One of the aspects that emerged as a significant strength 
was your adaptability. The second activity clearly wasn ’t working 
but you retrieved the situation with some deft adjustments 
crisply explained to your learners. This adaptability was evident 
in other aspects of the session. 

An example of a comment on a practice requiring further development 
might be: 

On two occasions your classroom management was rather 
diffident and uncertain. One was when the latecomer came 
in and interrupted the flow of the session with a somewhat 
dramatic entrance. The other was when one of the small groups 
doing that ‘first thoughts’ activity wandered from its task. 

You could have got the participants back on track sooner. 

The second approach suggests starting with a summary of strengths 
and weaknesses, using examples and illustrations from the session 
observed - ‘Your questioning techniques were particularly strong - 
I could identify questions at the higher levels of cognition requiring 
learners to respond in analytical and evaluative ways which signifi- 
cantly stimulated their learning’. Tutors can then be invited to add 
their own comments before a summary, as in the first approach, 
of what went well and what needs attention. 

The third approach starts with a chronological account of the session, 
going through good points and areas for attention. This can include 
( as with the other two approaches ) checking points with a tutor 
to ensure that in the complexity of the session the observer has 
not missed or misinterpreted a key point or episode. The tutor 
can comment and discuss during the chronology or at the end. 

The pattern then follows the other two approaches. 

Less experienced observers will find it useful to have these 
approaches to hand in their early observations and assessments, 
together with their observation forms and grading criteria, 
which will be mainly used at the summary stage. 
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More experienced observers may well use a combination of the 
approaches while covering the ground that needs to be covered. 
Both could usefully remember the mnemonic ‘COBS’. 

Feedback should be: 

n Clear 

■ Owned 

■ Balanced 

■ Specific. 

Activity 4 on conducting the observation, giving feedback and grading 

Arrange a 20-minute micro-teaching situation and feedback 
for ‘trainee observers’ to witness from beginning to end. 

The process will involve an observer watching a tutor, 
doing a report and giving feedback and a grade. 

Discuss the process at each stage with them. Establish any 
strengths and weaknesses observed and whether or not 
they agreed with the grade awarded. 

( Alternatively, go through the process up to where the observer 
awards a grade and then get the group to discuss which grade 
it should be.) 



Agreeing and implementing an action plan for 
organisational self-assessment and individual 
continuing professional development purposes 

Once an observation has been completed and a grade awarded 
( if you are going to have a grading system ), then an action plan 
can be agreed and completed between tutor and observer. This can 
include practicalities about the tutor’s performance, plans for general 
or specific training, and issues for the organisation to address 
(eg to do with rooming and resourcing). Figure 2 shows a completed 
action plan, and Appendix 1 contains a blank action plan. 
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Figure 2 Completed action plan following observation 

Action plan 

Agreed action, including staff development identified 



Action 


By whom 


By when 


i Attend/orthcoming 
Staff development event 
on 'learning styles’ 


K Jones 
(tutor) 


12/12/01 


2 Revamp handouts so 
they include more activities 
for 1 earners 


K Jones 
(tutor) 


March 2002 


3 Send, details of final part 
ofCertEd training 


R Miller 
(observer) 


14/12/01 


4 Attend forthcoming 
Staff development event 
on learning style ' 


K Jones 
(tutor) 


30/1 1/01 



Signed 

Observer R Miller 

Tutor K Jones Date 27/11/01 
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A well-established technique for deciding on action is 
to use SMART(ER) targets, that is targets that are: 

s Specific 
a Measurable 
h Achievable 
n Results-oriented 

■ Time-bound or time-specific 
m (Evaluative) 

■ (Reviewable). 

This is a natural end to the process of one observation, but also 
the beginning of a wider process. Many observations and grades 
(if you are going to have them) should lead not just to development 
of the individual but also to the dissemination of good practice 
through an institution. Good tutors can be invaluable as mentors 
and even develop their work and careers as observers themselves. 
Where weaknesses or areas for action are identified for individuals, 
further training or peer support can be planned. 

Observations can highlight issues beyond the tutor such as resources 
and facilities, or issues to do with curriculum and course management 
that might be identified as weaknesses in a particular area of learning. 
Similarly, if there are the same consistent weaknesses in 
the teaching practice of a group of tutors, this should go as 
an identified weakness into the appropriate part of the SAR. 
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The importance of a protocol in an OTL system 

It is important for you to establish a protocol for the process of 
OTL so that both tutors and observers are comfortable with it. 

What follows might be a starting point: 

Bi Observers will be briefed and trained, 
n Observations will involve 1 week’s notice beforehand. 
m Observations will last at least 45 minutes and not more than 1 hour. 

b Observers will complete the agreed documentation and 
provide constructive feedback. 

b ‘Tutors’ comments on the observation process will be welcomed. 

b The OTL system will be regularly reviewed as part 
of the self-assessment process. 

b Disputes over feedback or grades will go to the quality manager 
for appeal. 

a Observations will be based on confidentiality, courtesy and objectivity. 

b Students and staff will be respected at all times and prejudice 
and discrimination avoided at all times. 

There are other protocol issues. Should observation times be 
negotiated and is there a choice of observer? Are guidelines for 
feedback clearly understood by everyone? How are disagreements 
dealt with? What about anonymity? These points need to be resolved 
according to circumstances in different ACL providers. The ALI and 
LSC requirements for self-assessment do not preclude you using 
any approaches that improve teaching and therefore enhance 
the quality of learners’ experiences. 
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How a system of observation of teaching 
and learning relates to the SAR 

The section on action planning suggested how OTL can help an 
organisation as well as individual tutors to develop. In practical 
terms, you will need to decide how you use the information you 
get from your OTL system. The list below indicates one way 
it might be done. 

■ A team of observers completes the agreed quota of tutor observations 
over an agreed timescale and covering the appropriate areas 
of learning. 

m A system of moderation, perhaps with observers and moderators 
making some joint visits ( adding relentlessly to cost and 
stress factors ! ), establishes consistency of feedback and grading 
(if grading is used). 

a The strengths and weaknesses of tutors’ work are established 
by curriculum leaders and managers on the basis of the moderated 
observation reports - either on a geographical, centre basis or 
by areas of learning. 

a If grading is used, a profile of grades is established on the basis 
of the observations, again either on a centre basis or by areas 
of learning. 

a Either individual centres and partners produce provisional SARs 
with all the data required, including OTL data, which is then ‘fed into’ 
the provider’s SAR. 

a Or individual centres or partners give the provider the information 
required and the provider integrates the data into an SAR including 
grades, strengths and weaknesses relating to the areas of learning. 



This chapter has drawn heavily on the work of Dixon with Moorse 
(1998), Dixon with Walker (2000) and Nicholls (2000). 

These authors have produced comprehensive schemes, 
materials and ideas for the observation of teaching and learning 
in a college context, which is equally relevant in an ACL context. 
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Implementing a system of 
teaching and learning observation 




Introducing or adapting an OTL system 
in the ACL context 

A good and comprehensive system for observing teaching and 
learning is only part of the story. It has to be implemented skilfully, 
carefully and realistically so that it successfully and genuinely 
becomes the centrepiece of the self-assessment process. 

There are a number of very real concerns in the ACL sector about 
introducing or adapting a current OTL system. 

One of the most significant concerns is the resource implication. 
How will the whole thing be funded and managed ? Another is dealing 
with the diversity of ACL provision in particular localities. Some LEA 
partners will be operating different quality systems while others will 
have little in the way of quality assurance at all. FE college partners 
may already have sophisticated OTL procedures in place. 

How will this diversity be dealt with ? Another concern centres 
on the part-time and voluntary nature of much of the work in ACL. 
Introducing an OTL system will require skill and sensitivity so that it 
is accepted and embraced by all tutors in all centres who contribute. 

This chapter and the next cannot necessarily supply hard and 
fast solutions to these difficult issues, and the degree of access 
to various standards funds and local LSC financial support has not 
been completely finalised. However, the aim is to stimulate ideas 
and encourage approaches that will succeed. 
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Overview of change management 

Organisations face different types of change - strategic, structural, 
process-oriented or people-centred - or a combination of these. 
Implementing or adapting a new system for OTL is derived from 
a strategic shift imposed on the post-compulsory sector involving 
self-assessment in general and the new inspection arrangements 
that put the learner at the heart of the system. Although these 
overall arrangements are externally triggered ( in this case by 
new legislation ), instituting a new OTL system is about a change 
in process and procedures and about changes affecting people. 
Because of the level of the change, it is more likely to be successfully 
led by middle and ‘first line’ managers, especially managers with 
responsibility for quality and staff development, though importantly 
they must be fully backed and supported by the senior members 
of the organisation and those of its partners. 



Planning change 

Change should be planned and anticipatory -taking into account 
what may occur outside and inside an organisation and its partners. 
This way the organisation and its partners are more able to predict 
results and control events, and thus there is more chance of success. 
Reactive rather than proactive management can lead to disaster. 

A useful starting point for the change process is to devise a brief 
‘force field analysis’ consisting of factors working for change and 
factors working against. Applied to introducing an OTL system, 
a force field analysis might look as in Figure 3. 
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Figure 3 Force field analysis of factors working for and against 
a new OTL system in LEA ACL 
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This quite rudimentary approach at least clarifies some 
of the issues and challenges and allows you to judge 
the relative importance of each factor. 
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A stepped approach to change 

Adopting a stepped approach to change can usefully follow up 
the force field analysis. A five-step approach is shown in Figure 4. 

Figure 4 Diagram showing change in steps 
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One of the most important steps would be to establish in practical, 
costed terms what main target is required. A simple example is 
given in the Narnshire case study in the next chapter, and a form 
for you to use for your own planning is also given there, together 
with a planning checklist. 

There is no doubt that a stepped approach looks extremely attractive 
in theory. The reality of change is, however, ‘messy’, and there are 
likely to be considerable overlaps in each of the identified processes. 
At least proper planning and implementation can obviate the 
worst effects of unpredictability. 



Management styles, strategies and tactics 
in implementing change 

One particular style or a combination of styles can be used to 
introduce change such as a new OTL system. You will need 
to look at current styles, strategies used in the past and your 
organisational culture before deciding what approach to adopt 
in your circumstances. Consultation ( ‘we discuss, I decide’ ) is apt 
in many circumstances. A laissez-faire (‘you get on with it’ ) approach 
might be appropriate in an environment where each adult education 
service provider in an LEA is competent and autonomous. 

A democratic style ( ‘we discuss, we decide' ) can be effective in 
certain conditions. Where change needs to be rapid, an authoritarian 
(‘I decide’) position is perhaps the most appropriate, but though 
you might need to consider this approach in the short term, 
it probably goes against the spirit of self-assessment in any other 
circumstance, and will eventually be self-defeating because of that. 

Another approach is to look at change strategies. A rational strategy 
uses persuasion techniques on those subject to change, arguing that 
from observation and experience a particular course of action is 
clearly the most beneficial. ‘You know it makes sense; it's logical’ 
probably mostly sums up this approach. A ‘power-coercive’ strategy 
makes a change compulsory - ‘just do it, or else ! ’ A third approach 
is linked to the idea of a ‘learning organisation’ - that in order to 
embrace change, training, re-training and further training ( or more 
root-and-branch education) are required. In many cases of change, 
the reality is that a mixture of strategies can be identified and 
this may well be so in the situation you are managing. 
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Linked to the ideas of general management styles and change 
strategies is a framework based on ‘production’ and ‘people’ values 
combined with management effectiveness. ‘Impoverished management’ 
is when there is little concern for people or the achievement of tasks 
(‘production values’). The opposite of this is when managers empha- 
sise both task and people. ‘Country club management’ is when 
people values are stressed to the detriment of getting on with 
the job, whereas ‘autocratic task management’ places too much value 
on finishing the job and not enough on the welfare of those who do it. 
Managers do not fall automatically and permanently into one category 
or another. Contingency and circumstances influence style. 

The diagram below ( Figure 5 ) gives an impression of this framework. 



Figure 5 People and production values in managers linked 
to management effectiveness 

Adapted from Blake and Moulton, in Dixon 1997, page 66 



People values 



High (goal achievement) 




Valuing people highly but maintaining a strong focus on the tasks to be 
accomplished is likely to mean that goals are successfully achieved, 
but this is not necessarily always so. 

As far as change tactics are concerned, you could consider a 
number of options often linked with particular management styles. 

A ‘Trojan mouse’ approach introduces an apparently small-scale change 
into a whole organisation, which has great long-term consequences. 
For ACL, saying that a tutor will be observed once every 2 years seems 
innocuous but will have far-reaching consequences. Introducing change 
successfully into a unit or department of an organisation can create 
a momentum for change throughout, so if you have an outreach centre 
or a partner whose quality work is outstanding after your initial 
setting up, then you can use that model and the momentum it 
has produced elsewhere. Other tactics include objectives setting, 
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team development and sustained communication programmes 
so that there is full understanding of what is being aimed for. 
Offering skills development as an accompaniment to change, 
for example providing training to tutors who are to be observed, 
is often effective in securing commitment. Attention to contracts 
that incorporate requirements to undertake staff development, 
undergo classroom observation and to attend meetings is a more 
‘coercive’ means of facilitating change than a more ‘cooperative’ 
approach that may have applied previously. 



Resistance to change and how to overcome it 

Resentment, anxiety, dissatisfaction and insecurity are often 
associated with change. People can be observed responding to it 
along a negative-conditional-positive continuum, from ‘die-hards’ 
at the negative end through to pragmatists in the middle and advocates 
at the positive end. An experienced and long-serving tutor of floristry 
in an adult education centre may take a ‘die-hard’ position by stating 
unequivocally that she is not going to be affected by OTL and will only 
participate to comply at a very rudimentary level. A basic skills tutor 
might take the stance of an ‘ideologue’ - ‘an OTL system is against 
my beliefs and undermines my professionalism since I have spent 
many years training and building up my knowledge and expertise’. 

A ‘pragmatic’ creative writing tutor might submit to OTL - ‘I might as 
well do what I have to - it won’t hurt me and it might do some good’. 

A tutor-facilitator in an outreach group working with lone parents 
might advocate the OTL system - ‘I am new to all this and having 
my work observed will give me pointers about how I can improve 
and that will improve my confidence no end’. 

If the problems are identified, you should seek to apply 
the following techniques for overcoming resistance: 

h participation - where everyone is encouraged to join in 

b open, timely and accurate communication - 
to overcome fear and rumours 

n advance warning - to prevent unsettling shocks and surprises 
e sensitivity. 

Figure 6 amplifies these points. 
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Figure 6 Implementation of change - ideas for good practice applied to an OTL system 

Consultation 

e It should be real, comprehensive and seen to make a difference. If tutors make 
important contributions they should be incorporated into an OTL scheme. 

a The aim should be commitment and ‘ownership’ and not merely compliance. 

Communication 

a A ‘Quality’ newsletter and other communications should regularly circulate 
round LEA ALC services and partner organisations. 

a Full and open information about reasons for change should be given - 

and why senior managers are having to respond to a national ‘quality agenda’. 

a Effective interpersonal and information-giving skills can be used to try 
to overcome cynicism about consultation and change through a series of 
direct meetings to suit the arrangements of tutors (many of whom teach 
evening classes and work other unsocial hours ). 

People 

a Alliances should be made with the supporters of change in the centre of a 
service and in other sections - through volunteer observers and tutors for OTL. 

a Change has human consequences (stress, status, threat) and support 
should be offered - not all tutors and managers are confident about change. 

a Roles and responsibilities should be clear in the change process. 

a People working in flexible structures/systems can accommodate change 
more easily. 

a Staff development implications of change should not be ignored and 
there is likely to be a need for stage 1 and stage 2 teacher training. 
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in ACL 



Resources and costs 

b Resource and cost implications should not be overlooked - 
for organisations and for tutors participating in OTL. 

Aspects of change 

■ Introducing an OTL system will take time - this should be accepted. 

b Change can have a ripple effect on other systems and planning - 
there may be unexpected implications for ACL providers. 

a What is working well should be preserved (don’t ‘throw the baby out with the bathwater’). 
b Forward, long-term thinking should be encouraged. 

a The reason for change should never be forgotten - and first principles returned to 
as often as necessary. OTL is a major strand in improving the practice of tutors 
in LEA ACL services and their partners so that the quality of learners' experiences 
are always high. 

a The new or adapted system of OTL should be integrated with existing systems, 
a The new system must be evaluated, monitored and reviewed. 
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Sources in writing this chapter have been Plunkett and Attner(1994), 
Dixon (1997), Stewart (1998) and Peeke (1999). 
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A case study - Narnshire LEA 

The case study is presented in the form of an implementation pro- 
posal and schedule. It is stressed that this is not the only way or the 
best way that change can be achieved, since all circumstances are 
very different, but it should give insights into the possibilities available. 

The context 

The shire county of Narnshire is one of great contrasts. Of its popu- 
lation, 25% live in its large rural development area, 40% in its four 
market towns, and 35% in the town of Narnchester. Total population 
is around half a million; its 19+ population is just under 400,000. 

Narnchester is a large metropolitan district. It is divided into 26 wards, 
ranging from affluent wards in the central area to severely deprived 
areas in the east and west of the city. The adult population is declining. 
Narnchester narrowly missed becoming a unitary authority in 1997. 

Unemployment is estimated at 5.5% across the county and 7.6% 
in Narnchester. Approximately 10% of the population have some 
form of disability or long-term limiting illness. Narnchester has 
an ethnic minority population of 4.1% (the county figure is 1.5%). 

Educational attainment for school leavers is below the regional 
and national averages. Approximately 17.6% of adults have 
difficulty with reading and writing compared with 15% nationally. 
Most adult qualifications are achieved at NVQ level 2. 

Overall levels of participation in learning by adults are estimated 
as low; however, there is little accurate comparative data to 
confirm this. Current participation by enrolments in LEA-supported 
adult learning is just under 8% of the adult population. 
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Narnshire LEA 



In 2000 the county council identified raising educational achievement 
as its top priority. Although the primary focus has been on standards 
in statutory-age education, raising the skill levels of the adult popu- 
lation is an important dimension to achieving this priority. A new post 
of quality assurance manager has been established within the LEA’s 
Lifelong Learning Team with a general brief to manage self-assessment 
across LEA ACL provision and that of its partners, and with 
a specific brief to introduce a system of OTL. 

The Lifelong Learning division ( headed by an assistant director 
of education ) consists of four sections: Early Years Development 
and Childcare, Youth Service, Out of School Hours Learning, 
and Adult and Family Learning. 

Narnshire LEA delivers adult continuing education provision in part- 
nership with Narnchester’s FE sector college, LEA maintained schools, 
and a consortium of community schools, libraries and voluntary 
organisations. It makes direct provision for family learning, adult 
basic skills and ESOL ( English for speakers of other languages ) . 
LEA-supported adult continuing education provision is coordinated 
from the central Adult and Family Learning Team. 

The Service Manager is the responsible officer within the LEA for 
managing contracts from the local LSC and other external funders, 
and to providers of mainstream adult education. 
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Details are shown in Table 1. 



Table l Contract holders for mainstream adult education 



Provider 


Area covered 


Contract 


Enrolments 


Narnchester College 
of Further Education 


The Narnchester district 


£80,000 


12,000 


Brough Community School 


Broughville town 


£35,000 


3,000 


Tough Community College 


Toughville town 


£35,000 


2,500 


Lough Technology College 


Loughville town 


£35,000 


2,000 


Enough School 


Enoughville town 


£35,000 


1,500 


Vale of Slough Consortium 
of Community Colleges 


Rural development area 


£60,000 


6,000 


Total 


£280,000 


27,000 



Providers with contracts with the LEA recruit and directly manage 
staff who deliver the services (eg local managers, coordinators 
and tutors). These individuals may have other roles within 
the organisation they work for. 

In addition, the Adult and Family Learning Team commissions 
provision from small voluntary sector providers on a bid basis. 
This accounts for an additional 1500 learners per annum and 
costs £20,000. 
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Figure 7 Report to the assistant director of lifelong learning and partner managers 

Rationale for OTL system 

b The Learning and Skills Act 2000 requires all post-16 organisations delivering 
teaching and learning to produce annual SARs and development plans for 
local learning and skills councils. 

■ All these post-16 organisations and their contracted partners are subject 
to quadrennial inspections. 

a The new arrangements put the learner at the heart of everything. 
b Teaching and learning therefore have an even greater prominence than before. 

a A system of OTL is required for ACL in the authority and its contracted partners 
to prove the quality of their work and so that they can ‘continuously improve’. 

Goal 

To have a fully implemented and robust system of OTL operating 
in the LEA ACL service and partner organisations which will: 

a be understood, accepted and ‘owned’ by all staff in the LEA and 
partner organisations 

b build on the current LEA system and the expertise of the FE college partner, 
which already implements an OTL system 

b integrate other quality systems used by partner organisations 

b increase the professional expertise of tutors through developing their skills, 
knowledge and attitudes (especially their confidence, their application of 
equal opportunities policy and their receptivity to change) 

a improve the overall quality of provision in the LEA and its partners 
a contribute to the ethos and practice of continuous improvement 

a complement or be integrated with OTL systems currently operating 
in some providers 

b achieve observations of all new tutors within a term of starting and 
100% of all tutors over a 3-year period. 
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the introduction of a system of OTL to ACL services within the authority and its partners 

The main target 

All tutors in the ACL service (direct provision and partners) will be observed 
on one occasion each over a 3-year period. 



Table 2 Costing observation of teaching and learning 
and number of visits per observer 



Number of full-time tutors 


50 


Number of part-time tutors teaching over 180 hours per year 


150 


Number of part-time tutors teaching less than 180 hours per year 


150 


Number of trained observers 


20 


Estimated hours for each observation process 
(based on 1 hour actual observation and 
2 hours planning, including action planning) 


3 hours 


Overall cost per observation ( including travel costs ) 


£50 



Total tutors to be observed and number of visits: 350 

Total hours of observation: 

350 tutors x 3 hours per observation = 1050 hours 

Total cost @ £50 per hour = £52,500 

Cost over 3 years = £17,500 per year 

Total number of visits per observer = 17.5 

Approximate number of visits per year per observer = 6 

Cost of 6-month schedule for introducing OTL system 
in ACL services in addition to normal costs: £5000 



Note 

It is probably more sensible to overestimate the hourly cost of 
OTL activity to take into account travel, pre-observation activity, 
post-observation activity and 18% on-costs of any hourly rate, 
and therefore £50 per hour overall is not unrealistic. It is a good 
starting point for establishing that an OTL system is not going 
to be cheap! 
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Table 3 Six-month schedule for introducing a system for OTL in ACL services 



Organisation 
and partners 



Observers 



Tutors 



By 1 October 2001 

Quality manager presents schedule to head of lifelong learning 
and managers of partner organisations and gets approval 

Change leaders/'champions’ identified - curriculum leaders, 

VCO liaison staff, departmental staff, centre heads 

OTL committee set up (quality manager and change agents - 
meeting no.l) (FE college quality manager co-opted for 
SAR and OTL expertise) 

Quality manager visits all ACL centres and partner centres 

Observers identified, approved and selected according to 
established criteria (eg status in the organisation (course tutor/ 
leader or curriculum manager), teacher-trained, first degree or 
higher degree in education, tutor on C&G 730 or Certificate of 
Education programme, having D34 IV qualification) 

Tutors receive ‘Top Quality’ newsletter no. 1 covering: 

the new quality arrangements 

self-assessment - what it is and action organisation 

is required to make 

observation of teaching and learning 

call for volunteers for pilot. 



Handbook 



Research into structure and content includes : 
s schemes of work forms 
■ session plan forms 
b observation forms 
b protocols etc. 
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By 1 November 2001 



Organisation Self-assessment report in progress 
and partners visit an( j advice from ACL consultant 

Consultation on process (not on need for process or otherwise ! ) 

Self-assessment and OTL on agenda of every LEA and 
partners’ curriculum meetings 

Meeting no. 2 of OTL committee 
Change leaders start process in sub-units 
Identification of staff development needs 



Observers 



Tutors 



Handbook 



Training session no. 1 - the role of teaching and learning 
observation within the SAR process 

Tutors receive ‘Top Quality’ newsletter no. 2 covering: 

■ what to expect and what it means to tutors 
□ grading and the CIF 

m how to make your voice heard ( consultation on staff views ). 
Training for pilot volunteers 

General staff development day - equal opportunities, 
learning styles, classroom observation 

Development ( including the use of models from other organisations) 



By 1 December 2001 

Organisation Meeting no. 3 of OTL committee 

and partners Self-assessment and OTL on agenda of every LEA and partners’ 
curriculum meetings 
Self-assessment report drafted 

Start of implementation of staff development plan 
( including qualification for observers of teaching and learning) 

Consultation on the system for OTL and handbook 
Introduction of new contract for newly appointed tutors 
Observers Training session no. 2 - the OTL process, basic observation techniques 
Preparation for OTL pilot 

Tutors Tutors receive ‘Top Quality’ newsletter no. 3 covering : 

a the OTL handbook - overview and explanation, call for comments. 

Consultation questionnaire 

Training for pilot volunteers ( including preparation for pilot) 
Handbook Completion of first draft of handbook 
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Table 3 continued 





By 1 January 2002 


Organisation 
and partners 


Meeting no. 4 of OTL committee 

Self-assessment and OTL on agenda of every LEA 
and partners’ curriculum meetings 

Consultation feedback 
Pilot OTL starts 
Feedback from pilot 


Observers 


Training session no. 3 - OTL report writing and giving feedback 
Pilot OTL by teams of two observers 
Moderation meetings 

Feedback from pilot - observers’ viewpoint 


Tutors 

i 


Tutors receive ‘Top Quality’ newsletter no. 4 covering : 
a consultation feedback 



e volunteers’ participation in pilot 
b feedback from pilot - volunteers’ viewpoint. 



Handbook 


Second draft of handbook 

By 1 February 2002 


Organisation 
and partners 


Self-assessment and OTL on agenda of every LEA and 
partners’ curriculum meetings 

Meeting no. 5 of OTL committee 
OTL starts organisation wide 


Observers 


Observers start ‘caseloads’ - percentage of paired observations 
Moderation of OTL feedback and grades 


Tutors 


All tutors currently teaching receive one OTL assessment and grade 
Tutors receive ‘Top Quality’ newsletter no. 5 covering: 



b How was it for you? 



Handbook Third draft agreed and sent to printer 
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By 1 March 2002 

Organisation Review of process ( review conference ?) 
and partners Self-assessment and OTL on agenda of every LEA and 
partners’ curriculum meetings 
Analysis of OTL results 
Results to SAR and development plan 

Identification of staff development needs - 

observers (accreditation?), tutors (initial teacher training?) 

Review of process and ‘tweaking’ 

Observation schedule for following academic year 



Observers 


Review of process 

Percentages of pilot grades (1-7) into the SAR and 
development plan to inform next year’s OTL and SAR 


Tutors 


Review of process 




Tutors receive ‘Top Quality’ newsletter no. 6 covering: 
m What has been learnt? 




All new tutors observed 




30% of all tutors in ACL LEA services and 
partner organisations observed 


Handbook 


Final version of handbook valid for 12 months (till next review) 
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The case study and management 
of change theory 

The change in Narnshire LEA is ‘process’ and ‘people centred’ and 
therefore most appropriately led by the quality assurance manager 
with senior management support. You will notice that there is plenty 
of evidence of a stepped approach in the proposals to introduce an 
OTL system. The need for change is recognised in the rationale and 
the goal is clearly stated in eight different parts. The ‘change agents’ 
led by the quality manager are identified and generally the ‘method 
of intervention’ is strongly consultative. The importance of involving 
the quality manager at Narnchester College of Further Education is 
recognised by getting that person on the OTL committee. This person 
will have experience of self-assessment and a system of OTL and 
therefore the expertise required for the LEA. There is an implicit 
recognition that the change needs to be accepted by those who 
will be most directly affected. If change had been necessary on 
a shorter timescale then a more ‘authoritarian’ emphasis might 
have been required. 

At this stage of self-assessment in the ACL sector, evidence that it 
is being addressed and that a strong OTL system is in prospect is a 
valid approach as Narnshire LEA does not have a fully operating system. 
There may be elements of democracy in the process, where volunteers 
are requested for the pilot OTL scheme from all the different centres 
and partners - especially at the end where changes would result 
from review. An ‘authoritarian’ emphasis can be discerned in the 
introduction of contracts requiring attendance at meetings and at 
staff development events. Similarly, there might be little negotiation 
in establishing who should undertake the observations in the 
OTL system ( ie those with appropriate credibility conferred through 
qualifications and proven good practice), though a separate peer 
process could be much more flexible. The change is too complex and 
too important to allow a laissez-faire approach within outreach centres 
and partner organisations - and would be very patchy if it proceeded 
on these lines. Leadership is unequivocally required. 

You will probably agree that the proposed change in Narnshire’s 
schedule has a ‘rationalist’ feel to it with much communication 
offered through regular newsletters, curriculum meetings (where it 
is meant to be on the agenda every time), and a review conference. 
In addition, there is a strong ‘learning organisation’ orientation through 
the various staff development events for all those involved and 
even the offer of qualifications - for tutors and observers. 
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In terms of tactics, you probably get no sense of a small-scale change - 
it is a ‘big bang’ approach encompassing ACL services within the LEA 
and its partners. Similarly, change through a small unit as a model 
has not been adopted as an option. The emphasis on communication 
(through newsletters, meetings and training) is strong, as is 
target setting and team development. Introducing new contracts for 
new staff is a tactic that adds some steel to the implementation process. 
As Narnshire LEA and its partner organisations operate within 
‘role cultures’ ( ie hierarchies rather than more flexible webs, 
matrices or clusters ) this should not be a difficult issue with 
proper preparation and explanation. 

The generally consultative approach with its emphasis on participation 
by everyone, open, timely and honest communication, sensitivity 
and advance warning ought to overcome most resistance. 

The establishment of mutual trust, the ability of an organisation 
and its partners to learn and improve, and the ability to adapt should 
ensure success. Change is always recognised as being hard work 
requiring energy and commitment, but it is perhaps less stressful 
than a reactive approach. 

If change fails, it can be for a number of reasons. In the example 
of this case study, there is probably little evidence of ‘faulty thinking’ 
but if the various ‘milestones’ in the schedule are neglected or there 
is slippage ( ie loss of a ‘task’ focus ) then the process can go awry. 
The initial goals may also be over ambitious. 

As you read this and consider your own service and its partners, 
you may think that the key to the success of introducing an OTL system 
is resources and time. Will enough management time be allocated? 
Are there enough observers and potential observers for the system 
planned ? Is there extra funding for conferences and staff develop- 
ment? How will the hours for the whole observation process be 
funded ? Change does require proper investment of resources and, 
if they are not available, perhaps it is important to scale down 
the scope of the initial proposals so that funding them is 
a more modest undertaking. 
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Activity 5 A checklist for preliminary planning and a form for establishing targets for an 

When you have examined the case study in terms of change management 
theory, consider using the checklist below ( Figure 8) for preliminary planning 
of an OTL system in your organisation and its partners. 

Figure 8 Planning and implementing an OTL system in your LEA ACL service 

Refer to pages 19-28 and Figure 3 ( page 21 ), 4 ( page 22 ) and 9 ( page 43 ) 
to help you complete this form 



Following a 
‘force field analysis’, 
what are the main factors 
working for and against 
change in your service? 
(see Figure 3) 



Factors for 



Factors against 



Why is change required Reasons for change (why?) 
in your organisation 
and in your partners’ ? 

(see Figure 4) 



What goals are you Goals 
going to establish 

(what is to be achieved)? - 

(see Figures 4 and 9) 



How are you going Strategies 

to achieve the 
proposed change - 

what strategies, 

management styles Styles 

and tactics will you use - 

or what combinations 

of them (see Figure 4)? — 
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OTL system 



Who will be your 


Name 




‘change agents’ or 
‘change champions’ ? 
(see Figure 4) 


Designation and organisation 




Name 




Designation and organisation 






Name 






Designation and organisation 






Name 






Designation and organisation 






Name 






Designation and organisation 




Explain your 


When, where and how are things to develop? 




implementation plan 
(see Figure 4) 
















How will you monitor 






each stage of your 
implementation plan? 










(see Figure 4) 






Explain how you know 






your change procedure 
has been successful 










(evaluate the results) 






(see Figure 4) 







How do you intend 
to overcome possible 



resistance to your plan? _ 
(see pages 25-28) 
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Activity 6 How to achieve and cost a system of OTL 



Use Figure 9 to get some idea of how OTL can be achieved and 
how much it may cost. The possible targets in Figure 9 are 
not recommendations but to get you thinking about implications. 

Work out the additional scenarios with your organisation in mind, 
a Every tutor will be observed in the first year of her or his 
employment. 

b At least 30% of courses from every centre will have their tutors 
observed anually. 

a A sample of tutors from every centre will have tutors 
observed anually. 

a All observations will be carried out by staff trained as observers 
( this gives an idea, once the number of observations is established, 
of how many observations each observer will have to undertake). 

Now develop an OTL target and costings for your organisation. 



42 Observation of teaching and learning in adult education 



50 



Figure 9 Establishing and costing your initial target 
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1 Every tutor will be observed 
once in the next academic year 
d = 3 (a + b + c) 












2 All tutors in the ACL service 
(direct provision and partners) will 
be observed on one occasion each 
over a 3-year rolling programme 

3(a + b + c) 
d ( per annum ) — 












3 All staff who teach 180 hours 
per annum or more will be observed 
in the next academic year 
d = 3 (a + b) 













* Based on 1 hour actual observation and 2 hours planning including action planning - 
3 hours per tutor observed 

** Based on £50 per hour including travel and on-costs 
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As quality managers, you face many challenges in introducing or 
adapting a system of OTL to take into account the new arrangements. 
The booklet has established the context for having a system for OTL 
as part of the self-assessment and development planning process 
required by the new legislation. It has explained the process for 
OTL and the practicalities of doing it, including how OTL relates 
to the SAR. An overview of change management has led on to 
a case study to explain how the change required might be managed. 
We hope that this booklet has given you some guidelines and pointers 
about how you might proceed, and these will be backed up by all 
the services provided by the Adult and Community Learning 
Quality Support Programme. 
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Appendix i Buckinghamshire County Council OIL forms 
and guidance 







TEACHING OBSERVATION REPORT 




Course/group: 


Tutor: 


Observer: 


Session details: 


Session number/no. of sessions 


Date: 


Time of session: 


Number on register: Number present: 


Venue: 


Evidence of follow up of absent students: 








Records of students’ achievement: 










Planning 


Comments 





• clarity of aims/objectives/outcomes 

■ match of planned aims/objectives and ■ 



outcomes to learners 

• plans for individual and group learning: 

• informed by initial assessment 

• developed by tutors and learners 
together 

• reviewed and updated regularly 

• planning sufficiently detailed to aid 
effective delivery and learning 

• variety of delivery and assessment 
strategies planned 

• use made of records of progress and 

achievement to inform future planning 

Effectiveness of teaching and learning Comments 

• tutor's knowledge/competence contributes 
to effective delivery and assessment 

• learners understand objectives 

• methods and styles used are consistent 
with programme aims and learners’ 
personal objectives 

• learners are encouraged to develop the 
capacity to work independently 

» learners are acquiring new knowledge, 
developing ideas and increasing their 
understanding 

• learners understand what they are doing, 
how well they are doing, and how to 
improve 

• learners are challenged and inspired 

• evidence of good working relationships 
which foster learning 

• learners are active, apply effort to succeed, 
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Appendix 1 continued 



work productively and make effective use 
of time 

■ tutor presents material in a way which is 
sensitive to issues of equal opportunity 

■ resources promote effective teaching and 
learning 

■ ICT is used to enhance learning where 
appropriate 

■ health and safety is taken into account 




Assessment and monitoring of 
learners’ progress 


Comments 


■ assessment and recording are suitable for 
the course and the learners 

■ assignments set when appropriate and 
marked in a way that helps learners to 
progress 

■ assessment is fair, accurate and carried out 
regularly 

■ assessment and feedback ensure that 
learners know how well they are doing and 
what they need to do to improve 

■ the accreditation chosen is appropriate 


* 


Learner response and achievement 


Comments 


■ learners are expected to attend regularly 
and be punctual 

■ learners are stimulated and show interest 
and enthusiasm for learning and 
achievement 

■ learners are acquiring new knowledge, 
skills, understanding and developing their 
ideas 

• learners are developing skills of critical 
evaluation, research and analysis 

■ challenging learning goals arc achieved 

■ learners make significant progress towards 
achieving their goals 

■ learners' achievements are recorded 




General evaluation of session 


Comments 


• Were the objectives/outcomes suitably 
challenging for the learners and met by 
them? 

■ Were there any difficulties with learning 
identified? 

■ Were these overcome? 

• Further comments on the overall 
effectiveness of session 
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INDICATION OF SIGNIFICANT STRENGTHS AND WEAKNESSES 



Please record here notable strengths, and weaknesses that need positive action for 
improvement. Agreed action should be recorded on the next page 





Significant strengths 


Areas for development 


Planning 






Effectiveness of 
teaching and 
learning 






Assessment and 
monitoring of 
learners’ progress 






Learner response 
and achievement 






General evaluation 
of session 







Grade given by observer: 
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Appendix 1 continued 

ACTION PLAN 



Agreed action, including staff development identified 



Acdon 


By whom 


By when 









Signed: 

Observer: 

Tutor: Date: 
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OBSERVING TEACHING AND LEARNING 
GUIDANCE TO OBSERVERS 



General guidance 

Normally tutors will be observed by their own Tutor Coordinator, except where 

♦ it is impossible because of course times 

♦ another coordinator within the County Curriculum Area team has specific 
expertise which would be helpful in assessing the quality of teaching and learning 

♦ previous ‘history* suggests another member of staff may be more appropriate. 

A copy of your provisional timetable for observation visits should be given to the 
Chair of your Subgroup, once it has been agreed with the Deputy Consortium 
Manager. It should included tutor’s name, course, place, date and time (if the last two 
have yet to be confirmed, please indicate likely timing, eg 3rd/4th week February). 

You should not discuss an individual tutor, or visit, with another tutor. The record 
should only be used for QA and professional development purposes, in accordance 
with the procedures specified here and in the QA framework: if a class visit is made 
because there are concerns regarding a tutor’s capability, it should be made outside 
this QA framework. 

Before the visit you should 

♦ give tutors 2 weeks’ notice of your intention to visit (so that they can tell their 
students that you will be visiting) 

♦ send them a copy of the Information for Tutors, the 2001-2002 Observation 
Report, the Tutor Self-Evaluation form and the Tutor Comment sheet 

♦ agree when and how you will feedback to them: wherever possible, this should be 
done face to face, but it is acknowledged that telephone may have to be used in 
some cases - feedback should normally take place 7-10 days after the observation 

♦ confirm your likely level of involvement in the session (nb this should be as little 
as possible), the duration of the visit, and explore whether there will be an 
opportunity to talk to students 

♦ find out from them a little bit about the course and the student group, and check 
whether there is anything else that they wish you to know. 

During the visit 

♦ it is suggested that you take notes, and complete the form afterwards - so that you 
concentrate on actually ‘seeing’ as much as possible: you should keep your notes 

♦ any informal questioning of students should be done professionally, and in such a 
way that does not personalise issues to do with the tutor, or raises doubts in their 
minds about their course 
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Appendix 1 continued 



♦ remember that ‘you see what you look for’ - so glancing at the prompts on the 
Observation Report form occasionally might be useful 

♦ please ensure that you check that the tutor has a record of student achievement 
(for all courses - please refer to the Tutor Handbook for guidance given to tutors 
on record keeping) 

♦ please check that non-attendance has been followed up in line with the service 
policy. 

Completing the Observation Report 

♦ you do not need to comment about every prompt on the form - but do give the 
relevant example if you do observe evidence that relates to a prompt 

♦ other comments you may wish to make about good practice, or poor..., should be 
included in the section with the most relevant heading - it doesn’t matter if there 
is not actually a related prompt 

♦ you should only make judgements for which you have evidence - either observed, 
or documented 

♦ you will grade what you see - regardless of what you know about the tutor’s 
performance on other occasions (although it is suggested that you make an 
appropriate note in the ‘General evaluation’ section) 

♦ you should complete all sections of the form, except the general evaluation section 
at the end - this would allow you to record anything which arose from your 
feedback conversation which you thought appropriate to record (eg perhaps a high 
level of self-awareness of things which had not gone well, and an understanding 
of how they could be improved) 

♦ the grade should directly follow through from what you have recorded on the form 
- eg if you have recorded evidence of some really good practice alongside a few 
(not major) things which could have been better - you award a 2. 

During feedback 

♦ ensure that the tutor leaves you a completed Self-Evaluation form 

♦ complete and agree the Action Plan* together 

♦ ensure that you have both signed the forms 

♦ remind them to complete the Tutor Comment sheet and send it in. 

♦The Action Plan might include, for example, simple provision of information, 
identifying appropriate ongoing support, specific staff development needs: there is no 
need to produce a lengthy action plan for the sake of it! 

Hopefully it will never arise, but if the tutor should seriously dispute the grade, in 
spite of your careful recording of evidence, invite them to write to your Consortium 
Manager detailing their concerns. If necessary, the normal County Council Grievance 
Procedure would apply. 

You arc welcome to use an electronic version of the form: it is suggested that you try 
to keep the Teaching Observation Report as a maximum of four pages, and therefore 
if you expand one section you would probably contract another. 

Afterwards 

Please take two copies of the original Observation Report and Tutor Self-Evaluation 
form - retain one yourself for your Quality Assurance file, return one to the tutor for 
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their Personal Development and Learning Record (PDLR) file - and send the originals 
to the Continuing Education HR Officer at County Hall, for the tutor’s personnel file. 

Moderation of grade 

You will be asked to bring your copy to a designated Curriculum SubGroup meeting 
for moderation. 

Grading criteria 

Please note that Grades 1 and 2 should not be awarded for provision that meets 
standards which would normally be expected of all teachers. 

Particularly bear in mind the key evaluation questions from the draft Common 
Inspection Framework: 

How effective are teaching, training and learning? 

♦ How well teaching and training meet individuals’ needs and the course or 
programme requirements 

♦ How well learners learn and make progress 

How well do learners achieve? 

♦ Success in achieving challenging targets, including agreed learning goals and 
qualifications 

♦ The standards of learners’ work and their progress towards targets 

♦ The development of personal and learning skills 

How effective are the assessment and monitoring of learners 9 
progress? 

♦ The rigour of assessment 

♦ The uses of assessment in planning learning and monitoring progress 

Grade 1 excellent 

Grade 2 very good 

Grade 3 good 

Grade 4 satisfactory 

Grade 5 unsatisfactory 

Grade 6 poor 

Grade 7 very poor 

Further guidance on what are the characteristics of practice indicated by the above 
descriptors will be developed during the Autumn Term 2001 . 
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Appendix 1 continued 




TEACHING OBSERVATION - INFORMATION FOR TUTORS 

The purpose of observing and grading teaching is to: 

a) support professional development, through identifying staff development needs 
and opportunities for improving and developing practice 

b) provide evidence of the quality of our provision. 



Before the visit you should 

♦ be given 2 weeks’ notice of the intended visit 

♦ have had the opportunity to give the observer some information about the course 
and the students they will be observing 

♦ have agreed arrangements for receiving feedback after the visit 

♦ receive a copy of the (revised) Teaching Observation Record 2000-2001, a Tutor 
Self-Evaluation form and a Tutor Comment sheet. 

The following records should be made available to the observer when they visit 

♦ Course register 

♦ Course information sheet 

♦ Syllabus, or other course documentation if an externally accredited course (copy 
of document if a CROCN course) 

♦ Scheme of work 

♦ Session plan 

♦ Session plans/records of sessions which have already taken place 

♦ Records of students’ progress 

Afterwards 

Before receiving feedback, please complete the Tutor Self-Evaluation form - if you 
are not sure what to include in each section, you may find it helpful to look at the 
prompts on the Observation Report which the observer completes. 

During the feedback session, your observer will go through your self-evaluation with 
you, and go through their comments on the Observation Report, and then agree with 
you any appropriate action to follow up the observation. 
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After receiving feedback, please complete and return the Tutor Comment sheet. 



Grading criteria 

Please note that Grades l to 2 will not be awarded for provision that meets standards 
which would normally be expected of all teachers. 

Observers are particularly asked to bear in mind the key evaluation questions from the 
draft Common Inspection Framework: 

How effective are teaching, training and learning? 

♦ How well teaching and training meet individuals’ needs and the course or 
programme requirements 

♦ How well learners learn and make progress 

How well do learners achieve? 

♦ Success in achieving challenging targets, including agreed learning goals an 
qualifications 

♦ The standards of learners’ work and their progress towards targets 

♦ The development of personal and learning skills 

How effective are the assessment and monitoring of learners’ progress ? 

♦ The rigour of assessment 

♦ The uses of assessment in planning learning and monitoring progress 



Grade 1 


excellent 


Grade 2 


very good 


Grade 3 


good 


Grade 4 


satisfactory 


Grade 5 


unsatisfactory 


Grade 6 


poor 


Grade 7 


very poor 



Further guidance on what are the characteristics of practice indicated by the above descriptors 
will be developed during the Autumn Term 2001 . 
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Buckinghamshire Continuing Education 



OBSERVATION OF TEACHING AND LEARNING 
TUTOR SELF-EVALUATION 



Course / Group: 


Tutor: 


Observer: 


Venue: 


Date: 


Time of Session: 



Please record here what you felt to he notable strengths, and weaknesses that need positive 
action for improvement, in the session which was observed. 





Significant Strengths 


Areas for Development 


Planning 






Effectiveness of 
teaching and learning 






Assessment & 
monitoring of 
students’ progress 






Learner response and 
achievement 






General Evaluation 







Signed: 



Tuitor: Date: 
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Buckinghamshire Continuing Education 
Teaching Observation - Tutor Comment Sheet 2000-2001 




TEACHING OBSERVATION - TUTOR COMMENT SHEET 



Please return the completed form to Strategic Manager for Programmes & Quality at County 
Hall. Continuing Education,^ Floor County Hall, Aylesbury, Bucks HP20 1UZ - or hand it in 
to your Consortium Office to be forwarded. 



Tutor: 




Observed By: 




Course title /group: 




Consortium: 


Venue: 


Date of Observation: 


Time of Observation: 


Grade: 





Did you feel that the comments made by your assessor were fair? 
If no, please explain why 



Were there any problems using the forms? 

If yes, please explain 

How long did you spend talking to your assessor when he/she gave you feedback? 
Was this face to face, or by telephone? 

Has your teaching changed in any way since the observation? 



In what way, if any, could the observation have been more helpful? 



Signed: Date: 

Please add any further comments which you wish to add to the reverse of this sheet, 
including continuation of any of the above where necessary . 
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Namshire CC Adult and Community Learning Session Plan 



Appendix 2 continued 
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Appendix 3 Pre-observation meeting form 

Narnshire CC Adult and Community Learning 
Observation of Teaching and Learning 
Pre-observation planning meeting between tutor and observer 



Tutor 




Observer 




Time 




Date 





Details of session to be 
observed (date, time, location, 
session number) 







Background information 



| Checklist of what will be required 



Course register 




Course information sheet 




Syllabus or other course 
documentation 




Scheme of work 




Session plan 




Records of previous 
sessions 




Records of students’ 
progress 





Date, time and location for feedback (if 
not immediately post-session) 



Signed (tutor): 
Signed (observer): 
Date: 
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64 Observation of teaching and learning in adult education 



adult and community learning 
quality support programme 




Adult and community services need to meet 
the challenges of the new post-16 learning sector, 
which puts 'the learner at the heart of everything'. 

A system for observing teaching and learning 
is a powerful means of achieving this and 
raising quality throughout the sector. 

This guide describes the process of observing 
teaching and learning and how it can be managed. 
It is designed to inform senior managers, managers 
responsible for quality and curriculum leaders. 

It can also be used to support staff training and 
to help design internal documents and procedures. 
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